Issue1367
Created on 2009-06-04.06:12:08 by marcelotaube, last changed 2009-06-05.07:15:27 by marcelotaube.
Messages | |||
---|---|---|---|
msg4794 (view) | Author: Marcelo Taube (marcelotaube) | Date: 2009-06-04.06:11:39 | |
I am using the popen2.popen2 function to open a child and get a pipe to him. Unfortunatelly when i print to the pipe the child does not get any output. Unfortunatelly i dont have internet connection in the computer i work with so i am not able to submit exactly the same program that caused the error, however i will post here some similar code, as similar as my memory allows. <Code> 1: child_stdout, chld_stdin = popen("/usr/bin/tee") 2: child_stdin.write("This is a line\n") 3: child_stdin.flush() 4: out = child_stdout.readline() 5: print "from child:", out </code> The code above should print “from child: This is a line” because tee prints back its line it receives. The code works as it should in cpython but in jython it gets stucked in line 4 when trying to read. The problem in jython is that the pipe is not flushed to ‘tee’, ‘tee’ gets not input and consequently it does not print back any value. Thus trying to read from child_stdout blocks the program to the end of the days. I have been doing some debugging and the problem seems to happen also with subprocess.Popen Since I have not enough knowledge of jython and the Popen implementation I cannot tell exactly what is the problem. In spite of that, I still could understand some of the code and I see that the pipe is implemented as a python file wrapping an object of type org.python.core.io.StreamIO wrapping the output stream returned by ‘java.lang.Process.getOutputStream(..)’ . I copied-pasted that code and to a function of mine and tried to play with it. I arrived at the conclution that there are two places to flush. The jython file handler has a flush function and the java output stream has another flush function that never gets flushed, that means that even when the jython file is not buffered, output is never printed. This pseudo-code will do the work that the previous pseudo-code should have done: <code> procBuild = java.lang.ProcessBuilder(["/usr/bin/tee"]) proc = procBuild.start() proc_out = proc.getOutputStream() child_stdout = org.python.core.io.StreamIO (proc_out, True) #copied True from the Popen code child_stdin = org.python.core.io.StreamIO (proc.getInputStream() , True) child_stdin.write("This is a line\n") child_stdin.flush() proc_out.flush() #NOTICE THIS LINE!!! out = child_stdout.readline() print "from child:", out </code> |
|||
msg4795 (view) | Author: Philip Jenvey (pjenvey) | Date: 2009-06-04.07:34:16 | |
You're right, we definitely need to be flushing the OutputStream wrapped by StreamIO. I've fixed that in r6449 Did your extracted out example with the extra flush work? If so, jython trunk should be the equivalent I do fear that this example still might block due to our file implementation. Our readline reads in large chunks as an optimization (this is copied from py3k), and I think that could still deadlock the stream to the process. I would like to see the actual code you're trying Since the flush issue is solved I'm closing this issue. Feel free to log another if we need to continue pressing forward on this example |
|||
msg4797 (view) | Author: Marcelo Taube (marcelotaube) | Date: 2009-06-05.07:15:24 | |
Thanks. That is a fast answer, this will let me erase the workaround from my project when you release the next version of jython2.5. The exact example i was using is running Rpyc in jython so i can have the best of both worlds, the java api and the cpython api which is more complete that jython right now. This is mandatory for my project since i have some code which uses Tkinter to draw its GUI and rewritting it is not an option. So i installed the last jython and Rpyc. After some modifications to Rpyc i managed to make it load under jython but when I tried to make the jython and cpython communicate both process would freeze forever. It took me a long time till i tracked down the problem to the pipe implementation, at first i thought that the problem was inside Rpyc. Rpyc sends information between the two processes using the write()+flush() to send data, and read() to read it. It does not use directly print or readline, so i think it wont be a problem. In spite of that I think it is quite dangerous to make a process stuck because of an optimization, imagine you are implementing a unix shell in jython, the user writes 'ls' + <enter> but nothing happens. ציטוט Philip Jenvey: > Philip Jenvey <pjenvey@users.sourceforge.net> added the comment: > > You're right, we definitely need to be flushing the OutputStream wrapped > by StreamIO. I've fixed that in r6449 > > Did your extracted out example with the extra flush work? If so, jython > trunk should be the equivalent > > I do fear that this example still might block due to our file > implementation. Our readline reads in large chunks as an optimization > (this is copied from py3k), and I think that could still deadlock the > stream to the process. I would like to see the actual code you're trying > > Since the flush issue is solved I'm closing this issue. Feel free to log > another if we need to continue pressing forward on this example > > ---------- > resolution: -> fixed > status: open -> closed > > _______________________________________ > Jython tracker <report@bugs.jython.org> > <http://bugs.jython.org/issue1367> > _______________________________________ > |
History | |||
---|---|---|---|
Date | User | Action | Args |
2009-06-05 07:15:28 | marcelotaube | set | messages: + msg4797 |
2009-06-04 07:34:46 | pjenvey | set | status: open -> closed resolution: fixed messages: + msg4795 |
2009-06-04 07:02:18 | pjenvey | set | assignee: pjenvey nosy: + pjenvey |
2009-06-04 06:12:21 | marcelotaube | create |
Supported by Python Software Foundation,
Powered by Roundup